Thermal infrared (IR) cameras capture image wavelengths in the range of approximately seven to fourteen micrometers. A typical IR camera uses an infrared sensor (or detector) to detect infrared energy that is guided to the sensor through the camera's lens. IR cameras can be utilized for a variety of imaging applications including, but not limited to, passive motion detection, night vision, thermal mapping, health care, building inspection, surveillance, and the like. Recently, an attempt has been made in the IR industry to integrate IR cameras in advanced driver assistance systems (ADAS) and autonomous vehicle systems.
The application, and hence the type of camera, may depend on the infrared spectrum used. The infrared spectrum lies outside the visible light range and consists of a near infrared section (NIR), with wavelengths of 0.75-1.0 micrometers (μm); a short infrared section (SWIR) with wavelengths of 1.0-3.0 micrometers (μm); a mid-infrared section (MIR), with wavelengths of 3.0-5.0 μm; and a far-infrared (FIR) section, with wavelengths of 7.0-14.0 μm.
One type of FIR sensor is an uncooled sensor having a small form factor. Such sensors can typically be mass-produced using low-cost technology. In a typical arrangement, an uncooled sensor does not require a cryocooler for proper operation, but does require a shutter for frequent calibration. A shutter is a mechanical element placed between the lens and the FIR sensor for alternately blocking and exposing the sensor to infrared wavelengths. Generally, a shutter includes a flat-bladed flag, a sleeve, and an arm that connects the sleeve to the flag. The flag opens and closes at predefined time intervals.
Each detector within an FIR sensor may be configured to be sensitive to temperature changes that may result either from thermal radiation of the outside scene, i.e., the image of interest, or from the internal, or ambient, camera radiation, causing parasitic ambient drift. Since each detector has a slightly different responsivity to ambient temperature, the ambient drift pattern adds a random offset to each pixel that persists between different video frames and changes as a function of ambient temperature. Ambient drift removal is one of the crucial steps in proper image processing of thermal cameras.
The shutter is used during a flat-field correction (FFC) process to address this ambient drift. In an FFC process, the shutter presents a uniform temperature source to the FIR sensor. While imaging the flat-field source, the camera updates the offset correction coefficients, resulting in a more uniform image after the process is completed. The duration of the FFC process lasts a few hundred milliseconds, during which the image captured just prior to the shutter blocking the field of view is frozen until the FFC process is completed, when the shutter is reopened. This process must occur every few minutes for proper calibration.
While using a shutter may improve the quality and accuracy of the thermal image captured by an FIR sensor, having a blackout period lasting hundreds of milliseconds is unacceptable in certain applications. For example, using a shutter-based FIR camera in advanced driver assistance systems and autonomous vehicle systems can pose a high risk, as the camera must frequently shut off for an unacceptable amount of down time for driving situations. In addition, shutters include moving parts that wear out over time. This may cause a camera to malfunction during use and shorten the life of the camera.
The FIR camera designed for advanced driver assistance systems and autonomous vehicle systems should meet additional constraints other than safety. Such constraints include a small form factor, accurate and low latency image processing, and low-power consumption. As such, currently available FIR cameras, and in particular shutter-based FIR cameras, are not well adapted for automotive applications, as the presence of a shutter requires a larger camera body and additional energy to power the shutter movement.
While there are manufacturers who produce shutterless cameras where the ambient drift pattern is initially calculated during the camera manufacturing process, basing the correction merely on calibration performed during the manufacturing process often leads to inaccurate results. This is because the output of an FIR sensor, and hence the camera, depends on the current internal (ambient) camera radiation, also known as parasitic ambient drift. Without properly rectifying this ambient drift, the output image includes a residual unwanted noise pattern.
It would therefore be advantageous to provide a solution to correct fixed pattern noises including ambient drift in shutterless FIR cameras.